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ABSTRACT 

Background Plan-do-study-act (PDSA) cycles 
provide a structure for iterative testing of 
changes to improve quality of systems. The 
method is widely accepted in healthcare 
improvement; however there is little overarching 
evaluation of how the method is applied. This 
paper proposes a theoretical framework for 
assessing the quality of application of PDSA 
cycles and explores the consistency with which 
the method has been applied in peer-reviewed 
literature against this framework. 
Methods NHS Evidence and Cochrane 
databases were searched by three independent 
reviewers. Empirical studies were included that 
reported application of the PDSA method in 
healthcare. Application of PDSA cycles was 
assessed against key features of the method, 
including documentation characteristics, use of 
iterative cycles, prediction-based testing of 
change, initial small-scale testing and use of data 
over time. 

Results 73 of 409 individual articles identified 
met the inclusion criteria. Of the 73 articles, 47 
documented PDSA cycles in sufficient detail for 
full analysis against the whole framework. Many 
of these studies reported application of the PDSA 
method that failed to accord with primary 
features of the method. Less than 20% (14/73) 
fully documented the application of a sequence 
of iterative cycles. Furthermore, a lack of 
adherence to the notion of small-scale change is 
apparent and only 15% (7/47) reported the use 
of quantitative data at monthly or more frequent 
data intervals to inform progression of cycles. 
Discussion To progress the development of the 
science of improvement, a greater understanding 
of the use of improvement methods, including 
PDSA, is essential to draw reliable conclusions 
about their effectiveness. This would be 
supported by the development of systematic and 
rigorous standards for the application and 
reporting of PDSAs. 



INTRODUCTION 

Delivering improvements in the quality 
and safety of healthcare remains an inter- 
national challenge. In recent years, quality 
improvement (QI) methods such as plan- 
so-study-act (PDSA) cycles have been 
used in an attempt to drive such improve- 
ments. The method is v\^idely used in 
healthcare improvement; hov\^ever there is 
little overarching evaluation of hov\^ the 
method is appHed. This paper proposes a 
theoretical framev\^ork for assessing the 
quality of application of PDSA cycles and 
explores the quality and consistency of 
PDSA cycle application against this frame- 
v\^ork as documented in peer-reviev\^ed 
literature. 

Use of PDSA cycles in healthcare 

Despite increased investment in research 
into the improvement of healthcare, 
evidence of effective QI interventions 
remains mixed, v\^ith many systematic 
reviev\^s concluding that such interven- 
tions are only effective in specific set- 
tings. ^""^ To make sense of these findings, 
it is necessary to understand that deliver- 
ing improvements in healthcare requires 
the alteration of processes v\^ithin complex 
social systems that change over time in 
predictable and unpredictable v\^ays.^ 
Research findings highlight the influential 
effect that local context can have on the 
success of an intervention^ ^ and, as such, 
'single-bullet' interventions are not antici- 
pated to deliver consistent improvements. 
Instead, effective interventions need to be 
complex and multi-faceted^"^ ^ and devel- 
oped iteratively to adapt to the local 
context and respond to unforeseen obsta- 
cles and unintended effects. Finding 
effective QI methods to support iterative 
development to test and evaluate 
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interventions to care is essential for delivery of high- 
quality and high-value care in a financially constrained 
environment. 

PDSA cycles provide one such method for structur- 
ing iterative development of change, either as a stan- 
dalone method or as part of v\^ider QI approaches, 
such as the Model for Improvement (MFI), Total 
Quality Management, Continuous QI, Lean, Six 
Sigma or 'Quality Improvement Collaboratives'.^ 
Despite increased use of QI methods, the evidence 
base for their effectiveness is poor and under- 
theorised. ^^"^^ PDSA cycles are often a central com- 
ponent of QI initiatives, hov\^ever fev\^ formal objective 
evaluations of their effectiveness or application have 
been carried out.^^ Some PDSA approaches have been 
demonstrated to result in significant improvements 
in care and patient outcomes, v\^hile others have 
demonstrated no improvement at all.^^~^^ 

Although at the surface level these results appear 
disheartening for those involved in QI, there is a need 
to explore the extent to v\^hich the PDSA method has 
been successfully deployed to drav\^ conclusions from 
these studies. Rather than see the PDSA method as a 
'black box' of QI,^^ it is important to understand that 
the use of PDSA cycles is, itself, a complex interven- 
tion made up of a series of interdependent steps and 
key principles that inform its application^ and 
that this application is also affected by local context. 
To interpret the results regarding the outcome (s) from 
the apphcation of PDSA cycles (eg, v\^hether processes 
or outcomes of care improved) and gauge the effect- 
iveness of the method, it is necessary to understand 
how the method has been appHed. 

No formal criteria for evaluating the application or 
reporting of PDSA cycles currently exist. It is only in 
recent years, through SQUIRE guidelines, that frame- 
v\^orks for pubUcation have been developed that expli- 
citly consider description of PDSA application.^^ 
We consider that such criteria are necessary to 
support and assess the effective application of PDSA 
cycles and to increase their legitimacy as a scientific 
method for improvement. We revisited the origins and 
theory of the method to develop a theoretical frame- 
v\^ork to evaluate the application of the method. 

The origins and theory of PDSA cycles 

The PDSA method originates from industry and 
Walter Shev\^hart and Edv\^ard Deming's articulation of 
iterative processes w^hich eventually became knov\^n as 
the four stages of PDSA.^^ PDCA (plan-do-check- 
act) terminology v\^as developed follov\^ing Deming's 
early teaching in Japan.^^ The terms PDSA and PDCA 
are often used interchangeably in reference to the 
method. This distinction is rarely referred to in the lit- 
erature and for the purpose of this article we consider 
PDSA and PDCA but refer to the methodologies gen- 
erally as 'PDSA cycles unless otherv\^ise stated. 



Users of the PDSA method follov\^ a prescribed four- 
stage cyclic learning approach to adapt changes aimed at 
improvement. In the 'plan' stage a change aimed at 
improvement is identified, the 'do' stage sees this 
change tested, the 'study' stage examines the success of 
the change and the 'act' stage identifies adaptations and 
next steps to inform a nev\^ cycle. The MFI^^ and 
FOCUS^^ (see figure 1) framev\^orks have been devel- 
oped to precede the use of PDSA and PDCA cycles^^ 
respectively (table 1). 

In comparison to more traditional healthcare 
research methods (such as randomised controlled 
trials in v\^hich the intervention is determined in 
advance and variation is attempted to be eliminated 
or controlled for), the PDSA cycle presents a prag- 
matic scientific method for testing changes in complex 
systems.^^ The four stages mirror the scientific experi- 
mental method^^ of formulating a hypothesis, collect- 
ing data to test this hypothesis, analysing and 
interpreting the results and making inferences to 
iterate the hypothesis. 

The pragmatic principles of PDSA cycles promote 
the use of a small-scale, iterative approach to test 
interventions, as this enables rapid assessment and 
provides flexibility to adapt the change according to 
feedback to ensure fit-for-purpose solutions are devel- 
oped.^^ Starting v\^ith small-scale tests provides 
users v\^ith freedom to act and learn; minimising risk 
to patients, the organisation and resources required 
and providing the opportunity to build evidence for 
change and engage stakeholders as confidence in the 
intervention increases. 

In line v\^ith the scientific experimental method, the 
PDSA cycle promotes prediction of the outcome of a 
test of change and subsequent measurement over time 
(quantitative or qualitative) to assess the impact of an 
intervention on the process or outcomes of interest. 
Thus, learning is primarily achieved through interven- 
tional experiments designed to test a change. In recog- 
nition of v\^orking in complex settings v\^ith inherent 
variability, measurement of data over time helps 
understand natural variation in a system, increase 
av\^areness of other factors influencing processes or 
outcomes, and understand the impact of an 
intervention. 

As v\^ith all scientific methods, documentation of 
each stage of the PDSA cycle is important to support 
scientific quality, local learning and reflection and to 
ensure knov\^ledge is captured to support organisa- 
tional memory and transferability of learning to other 
settings. 

This reviev\^ examines the application of PDSA 
cycles as determined by these principle features of the 
PDSA method described above. We recognise that a 
number of health and research related contextual 
factors may affect application of the method but these 
factors are beyond the scope of this review. The 
reviev\^ intends to improve the understanding of 
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What are we trying to accomplish? 



How will we know if a change is an 
improvement? 



What change can we make that will 
result in an improvement? 




Find a process to improve 



Organise a team that knows the process 



Clarify current knowledge of the process 



Understand causes of process variation 



Select the process improvement 

^ 



/ Act 


Plan \ 


\ Check 


Do / 



Figure 1 The Model for Improvement; FOCUS. 



whether the PDSA method is being used and reported 
in hne with the hterature informed criteria and there- 
fore inform the interpretation of studies that have 
used PDSA cycles to faciUtate iterative development of 
an intervention. 

METHODS 

A systematic narrative review was conducted in adher- 
ence to the Preferred Reporting Items for Systematic 
Reviews and Meta-Analyses (PRISMA) statement.^"^ 

Search 

The search was designed to identify peer-reviewed 
publications describing empirical studies that applied 
the PDSA method. Taking into account the develop- 
ment of the method and terminology, the search 
terms used were 'PDSA, TDCA, 'Deming Cycle', 
'Deming Circle', 'Deming Wheel' and 'Shewhart 
Cycle'. No year of publication restrictions were 
imposed. 

Information sources 

The following databases were searched for articles: 
Allied and Complementary Medicine Database 
(AMED; 1985 to present), British Nursing Index (BNI; 
1985 to present). Cumulative Index to Nursing and 
AUied Health Literature (CINAHL; 1981 to present), 
Embase (1980 to present). Health Business Elite 
(EMBESCO Publishing, Ipswich, Massachusetts, USA), 
the Health Management Information Consortium 
(HMIC), MEDLINE from PubMed (1950 to present) 
and PsychlNFO (1806 to present) using the NHS 
Evidence onUne library (REE), and the Cochrane 



Database of Systematic Reviews. The last search date 
was 25 September 2012. 

Data collection process and study selection 

Data were collected and tabulated independently by 
MJT, CM and CN in a manner guided by the 
Cochrane Handbook. Eligibility was decided inde- 
pendently, in a standardised manner and disagree- 
ments were resolved by consensus. If an abstract was 
not available from the database, the full-text reference 
was accessed. 

Inclusion criteria for articles were as follows: pub- 
hshed in peer-reviewed journal; describes PDSA 
method being applied to improve quality in a health- 
care setting; pubUshed in EngUsh. Editorial letters, 
conference abstracts, opinion and audit articles were 
excluded from the study selection. 

Data items 

A theoretical framework was constructed by compart- 
mentalising the key features of the PDSA method into 
observable variables for evaluation (table 2). This 
framework was developed in accordance with recom- 
mendations for PDSA use cited in the literature, 
describing the origins and theory of the method. Face 
validity of the framework was achieved through dis- 
cussion among authors, with QI facilitators and at 
local research meetings. 

Data were collected regarding application of the 
PDSA method in line with the theoretical framework. 
Other data collected included first author, year of 
publication, country, area of healthcare, use of PDSA 
or PDCA terminology, and use of MFI or FOCUS as 
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Table 1 Description of the plan-do-study-act (PDSA) cycle method according to developers and commentators 



Deming (1986)^^Origindl description of 
the method relating to manufacturing 



Langley (1996)^°How the PDSA method 
may be adapted for use in healthcare 
contexts 



Speroff and O'Connor (2004)^^How the 
PDSA method is analogous to scientific 
methodology 



Plan Plan a change or test aimed at improvement 



Do Carry out the change or test (preferably on 
a small scale) 



Study Examine the results. What did we learn? 
What went wrong? 



Act Adopt the change, abandon it or run 
through cycle again 



► Identify objective 

► Identify questions and predictions 

► Plan to carry out the cycle (who, when, 
where, when) 

► Execute the plan 

► Document problems and unexpected 
observations 

► Begin data analysis 

► Complete the data analysis 

► Compare data to predictions 

► Summarise what was learnt 

► What changes are to be made? 

► What will the next cycle entail? 



Formation of a hypothesis for improvement 



Conduct study protocol with collection of data 



Analysis and interpretation of the results 



Iteration for what to do next 



supporting frameworks. Ratios were used to analyse 
the results regarding the majority of variables, and 
mean scores regarding data associated with length of 
study, length of PDSA cycle and sample size were also 
used for analysis. Data were analysed independently 
by MJT and CM. Discrepancies (which occurred in 
less than 3% of data items) were resolved by 
consensus. 



Risk of bias across studies 

Despite our review being focused on reported appHca- 
tion, rather than success of interventions, it may still 
be possible that publication bias affected the results of 
this study. Research that used PDSA methodology, but 
did not yield successful results, may be less Hkely to 
get pubhshed than reports of successful PDSA 
interventions. 



Risk of bias in individual studies 

The present review aimed to assess the reported appH- 
cation of the PDSA method and the results of individ- 
ual studies were not analysed in this review. 



RESULTS 

Study selection 

A search of the databases yielded 942 articles. After 
removal of duplicates, 409 remained; 216 and 120 



Table 2 Theoretical framework based on key features of the plan-do-study-act (PDSA) cycle nnethod 
Feature of PDSA Description of feature 



How this was measured 



Iterative cycles 



Prediction-based test 
of change 

Small-scale testing 



Use of data over time 



Documentation 



To achieve an iterative approach, multiple PDSA cycles must 
occur. Lessons learned from one cycle link and inform cycles that 
follow. Depending on the knowledge gained from a PDSA cycle, 
the following cycle may seek to modify, expand, adopt or 
abandon a change that was tested 



A prediction of the outcome of a change is developed in the 
'plan' stage of a cycle. This change is then tested and examined 
by comparison of results with the prediction 

As certainty of success of a test of change is not guaranteed, 
PDSAs start small in scale and build in scale as confidence grows. 
This allows the change to be adapted according to feedback, 
minimises risk and facilitates rapid change and learning 

Data over time increases understanding regarding the variation 
inherent in a complex healthcare system. Use of data over time is 
necessary to understand the impact of a change on the process or 
outcome of interest 

Documentation is crucial to support local learning and 
transferability of learning to other settings 



► Were multiple cycles used? 

► Were multiple cycles linked to one another (ie, does 
the 'act' stage of one cycle inform the 'plan' stage of 
the cycle that follows)? 

► When isolated cycles were used were future actions 
postulated in the 'act' stage? 

► Was a change tested? 

► Was an explicit prediction articulated? 

► Sample size per cycle? 

► Temporal duration of cycles? 

► Number of changes tested per cycle? 

► Did sequential cycles increase scale of testing? 

► Was data collected over time? 

► Were statistics used to test the effect of changes 
and/or understand variation? 

► How thoroughly was the application of the PDSA 
method detailed in the reports? 

► Was each stage of the PDSA cycles documented? 
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were further discarded following review of abstracts 
and full texts, respectively. Excluded articles did not 
apply the PDSA method as part of an empirical study 
or coincidently used the acronyms PDSA or PDCA for 
different terms, or were abstracts for conferences or 
poster presentations. A total of 73 articles met the 
inclusion criteria and were included in the review (see 
figure 2). 

General study characteristics 

Country of study 

The retrieved articles describe studies conducted in 
the USA (n=46), the UK (n=13), Canada (n=3) 
Australia (n=3), the Netherlands (n=2) and one each 
from six other countries (see onUne supplementary 
appendix A for complete synthesis of results). 

Healthcare discipline to which method was applied 
This varied across acute and community care and clin- 
ical and organisational settings. The most common 
settings were those of pain management and surgery 
(six articles each). 

Method terminology 

Of the 73 articles identified, 42 articles used 'PDSA 
as terminology and 31 referred to the method as 
'PDCA. Eight of these reported using the MFI. 
Thirty-one articles used 'PDCA terminology, with 20 
using the preceding FOCUS framework. One article 
described use of FOCUS and MFI. Over time there 
was an increase in the prevalence of PDSA use with 



942 records identified 
through database 
searching 



I 



409 of records after duplicates 
removed 



I 



409 records screened 



I 



eened 



216 records 
excluded 



193 full-text articles 
assessed for eligibility 



I 



73 studies included in 
qualitative synthesis 



120 articles excluded that 
were not peer-reviewed 
articles describing use of PDSA 
methodology for quality 
improvement in healthcare 



Figure 2 PRISMA diagram. 



PDCA use diminishing (see online supplementary 
figure SI). The earliest reported use of PDCA and 
PDSA in healthcare was 1993 and 2000, respectively. 

Documentation 

The following four categories were used to describe 
the extent to which cycles were documented in arti- 
cles (n=73): no detail of cycles (n=16); themes of 
cycles (but no additional details) (n=8); details of 
individual cycles, but not of stages within cycles 
(n=8); details of cycles including separated informa- 
tion on stages of cycles (n=41). 

Analysis of articles against the developed framework 
was dependent on the extent to which the application 
of PDSA cycles was reported. Articles that provided 
no details of cycles or only themes of cycles were 
insufficient for full review and excluded for analysis 
against all features. Articles that provided further 
details of cycles completed (n=49) were included for 
analysis against the remaining four features of the 
framework. A full breakdown of findings can be 
viewed in online supplementary appendix B. 

Application of metliod 

Iterative cycles (n=49) 

Fourteen articles described a sequence of iterative 
cycles (two or more cycles with lessons learned from 
one cycle linking and informing a subsequent cycle), 
33 described isolated cycles that are not Hnked, and 2 
articles described cycles that used PDSA stages in the 
incorrect order (in one article, one plan, one do, two 
checks and three acts were described, PDACACA^^; a 
further study did not report use of a 'check' stage; 
PDA^^) and are excluded from further review. Of the 
33 articles that described non-iterative cycles, 29 
reported a single cycle being used, and 4 described 
multiple, isolated (non-sequential) cycles. Although 
future actions are often suggested in articles that 
reported a single cycle, only three explicitly men- 
tioned the possibihty of further cycles taking place. A 
total of 13.6% (3/22) of PDCA studies described the 
application of iterative cycles compared with 44% 
(11/25) of PDSA studies describing the appHcation of 
iterative cycles (see figure 3). 

Prediction-based testing of change (n=47) 

The aims of the cycles adhered to one of two themes: 
tests of a change; and collection or review of data 
without a change made. Of the 33 articles with single 
cycles, 30 aimed to test a change while 3 used the 
PDSA method to collect or review data. Of the 14 
articles demonstrating sequential cycle use, 8 solely 
used their cycles to test change whilse5 began with a 
cycle collecting or reviewing data followed by cycles 
testing change. One article described a mixture of 
cycles testing changes and cycles that involved collec- 
tion/review of data. Four of the 47 studies contained 
an explicit prediction regarding the outcome of a 
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■ Isolated cycles ■ Iterative cycles 



120.0% 




PDCA(N=22) PDSA(N = 25) Combined total (N - 47) 

Method terminology 

Figure 3 Iterative nature of cycles for all articles and split by 
plan-do-check-act and plan-do-study-act terminology. 

change; all 4 aimed to test a change (see onUne sup- 
plementary table SI). 

Small-scale testing (n=47) 

Scale was assessed in three ways: sample size, duration 
and complexity. Sample size refers to quantity of 
observations used to measure the change; duration 
refers to the length of PDSA cycle application; and 
complexity refers to the quantity of changes adminis- 
tered per cycle. 

Sample size 

Patient data, staff data and case data were used as 
samples within PDSA cycles. Twenty-seven articles 
reported a sample size from at least one of their 
cycles. Twenty-one of these were isolated cycle studies 
with sample size ranging from 7 to 2079 
(mean =3 23.3 3, SD =533.60). The remaining six 
studies reporting individual cycle sample sizes used 
iterative cycles; the sample size of the first cycles of 
these ranged from 1 to 34 (mean =16.75, SD = 11.47). 
Two of these studies described the use of incremental 
sample sizes across cycles, three used non-incremental 
sample sizes across cycles, and one changed the type 
of sample. Of the eight iterative cycle articles that did 
not report individual cycle sample sizes, two did not 
differentiate sample sizes between cycles and instead 
gave an overall sample for the chain of cycles and six 
did not report sample size. 

Duration 

Reported study duration of isolated cycles ranged 
from 2 weeks to 5 years (mean= 11.91 months, 
SD = 12.81). Only five articles describing iterative 
cycles explicitly reported individual cycle duration. 
Individual cycle duration could be estimated from the 
total duration of the PDSA cycle chain and the 
number of cycles conducted, resulting in approximate 
cycle lengths ranging from three cycles in 1 day to 
one cycle in 16 months (mean=5.41 months, 
SD=4.80, see online supplementary figure S2). The 
total PDSA cycle duration for series of iterative cycles 



(first to last cycle of one chain) ranged from 1 day to 
4 years (mean=20.38, SD=20.39 months). 

Complexity 

Twenty-two articles reported more than one change 
being tested within a single cycle. Of the articles 
describing iterative cycles, 42% administered more 
than one change per cycle compared with 48% of the 
articles describing non-iterative PDSA cycles. 

Data over time (n=47) 

All studies used a form of qualitative and quantitative 
data to assess cycles. Studies were categorised accord- 
ing to four types of reporting quantitative data: 
regular (n=15), three or more data points with con- 
sistent time intervals; non-regular (n=16), before and 
after or per PDSA cycle; single data point (n=8), a 
single data point after PDSA cycle(s); and no quantita- 
tive data reported (n=8). Of the 15 articles that used 
regular data, only 7 used monthly or more frequent 
data intervals (see onUne supplementary figure S3 for 
full frequency of regular quantitative data reporting). 
No studies reported using statistical process control to 
analyse data collected from PDSA cycles. Eleven 
included analysis of data using inferential statistical 
tests (five of these studies collected isolated data, six 
involved continuous data collection). 

Of the eight articles that did not report any quanti- 
tative data, two reported that quantitative analyses 
had taken place but did not present the findings and 
six described the use of qualitative feedback only (one 
non-regular, five single data point). Qualitative data 
were gathered through a range of mechanisms from 
informal staff or patient feedback to structured focus 
groups. 

DISCUSSION 

PDSA cycles offer a supporting mechanism for itera- 
tive development and scientific testing of improve- 
ments in complex healthcare systems. A review of the 
historic development and rationale behind PDSA 
cycles has informed the development of a theoretical 
framework to guide the evaluation of PDSA cycles 
against use of iterative cycles, initial small-scale 
testing, prediction-based testing of change, use of data 
over time and documentation. 

Using these criteria to assess peer-reviewed pubUca- 
tions of PDSA cycles demonstrates an inconsistent 
approach to the application and reporting of PDSA 
cycles and a lack of adherence to key principals of the 
method. Only 2/73^^ articles demonstrated compli- 
ance with criteria in all five principles. Assessment of 
compliance was problematic due to the marked vari- 
ation in reporting of this method, which reflects a 
lack of standardised reporting requirements for the 
PDSA method. 

From the articles that reported details of PDSA 
cycles it was possible to ascertain that variation is 
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inherent not just in reporting standards, but in the 
conduct of the method, implying that the key princi- 
ples of the PDSA method are frequently not followed. 
Less than 20% (14/73) of reviewed articles reported 
the conduct of iterative cycles of change, and of these, 
only 15% (2/14) used initial small-scale tests with 
increasing scale as confidence in the intervention 
developed. These results suggest that the full benefits 
of the PDSA method would probably not have been 
realised in these studies. Without an iterative 
approach, learning from one cycle is not used to 
inform the next cycle, and therefore it is unlikely that 
interventions will be adapted and optimised for use in 
a particular setting. Furthermore, large-scale cycles 
risk significant resource investment in an intervention 
that has not been tested and optimised within that 
environment and risk producing 'false' negatives. 

Only 14% (7/47) of articles reported use of regular 
data over time at monthly or more frequent intervals, 
indicating a lack of understanding around the use of 
the PDSA method to track change within a 'live' 
system, and limiting the ability to interpret the results 
from the study. Cycles that included an explicit predic- 
tion of outcomes were reported in only 9% (4/47) of 
articles, suggesting that PDSA cycles were not used as 
learning cycles to test and revise theory-based 
predictions. 

Overall these results demonstrate poor compliance 
with key principles of the PDSA method, suggesting 
that it is not being used optimally. The increasing 
trend in using PDSA (as opposed to 'PDCA') cycles in 
recent years, however, does seem to have been accom- 
panied by an increase in compliance with some key 
principles, such as use of iterative cycles. Deming was 
cautious over the use of the 'PDCA terminology and 
warned it referred to an expHcitly different process, 
referring to a quality control circle for dealing with 
faults within a system, rather than the PDSA process, 
which was intended for iterative learning and 
improvement of a product or a process.^^ This subtle 
difference in terminologies may help to explain the 
better compliance with key methodological principles 
in studies that refer to the method as 'PDSA. 

One of the articles identified in the search included 
comments by the authors that the PDSA method 
should be 'more realistically represented',"^^ as inef- 
fective cycles can be 'abandoned' early on, making it 
needless to go through all four stages in each iteration. 
These comments may provide insight into an import- 
ant potential misunderstanding of the PDSA method- 
ology. Ineffective changes will result in learning, 
which is a fundamental principle behind a PDSA 
cycle. However minor this abandoned trial may have 
been, it can still be usefully described as a PDSA cycle. 
A minor intervention may be planned (P) and put into 
practice (D). A barrier may be encountered (S), result- 
ing in a decision being made to retract the interven- 
tion, and to do something differently (A). 



The theoretical framework presented in this paper 
highlights the complexity of PDSA cycles and the 
underpinning knowledge required for correct appHca- 
tion. The considerable variation in application 
observed in the reported literature suggests that 
caution should be taken in interpreting results from 
evaluations in which PDSAs are used in a controlled 
setting and as a 'black box' of QI. This review did not 
compare the effectiveness of use to reported outcomes 
and therefore this study does not conclude whether 
better appHcation of the PDSA method results in 
better outcomes, but instead draws on theoretical 
principles of PDSAs to rationaUse why this would be 
expected. Prospective mechanistic studies exploring 
the effective application of the method as well as 
study outcomes would be of greater use in drawing 
conclusions regarding the effectiveness of the method. 
The framework presented in this paper could act as a 
good starting point for such studies. 

The fact that only peer-reviewed publications were 
assessed in this study means that results may be 
affected by publication bias. This is anticipated both 
in terms of what is accepted for publication but also 
the level and type of detail that is requested and 
allowed in typical publications (eg, before and after 
studies are more common than presenting data over 
time and this may make these types of studies easier 
to pubhsh). Though QI work may be easier to pubUsh 
now through recent changes in publication guide- 
hnes,^^ possible publication outlets continue to be 
relatively limited. 

To support systematic reporting and encourage 
appropriate usage, we suggest that reporting guidelines 
be produced for users of the PDSA method to increase 
transparency as to the issues that were encountered and 
how they were resolved. While PDSA is analogous to a 
scientific method, it appears to be rarely used or 
reported with scientific rigour, which in turn, inhibits 
perceptions of PDSA as a scientific method. Such 
guidelines are essential to increase the scientific legit- 
imacy of the PDSA method as well as to improve scien- 
tific rigour or application and reporting. Although the 
SQUIRE guidelines make reference to the potential use 
of PDSA cycles, further support to users and teachers, 
and publication of this improvement method seems 
necessary. Consistent reporting of PDSA structure 
would allow meta-evaluation and systematic reviews to 
further build the knowledge of how to use such 
methods effectively and the principles to apply to 
increase chances of success. 

It is clear from these findings that there is much 
room for improvement in the application and use of 
the PDSA method. Previous studies have discussed the 
influence of different context factors on the use of QI 
methods, such as motivation, data support infrastruc- 
ture and leaderships^ 41-43 understanding how 
high-quality usage can be promoted and supported 
needs to become the focus of further research if such 
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QI methods are going to be used effectively in main- 
stream healthcare. 

CONCLUSIONS 

There is varied application and reporting of PDSAs 
and lack of compliance v\^ith the principles that under- 
pin its design as a pragmatic scientific method. The 
varied practice compromises its effectiveness as a 
method for improvement and cautions against studies 
that viev\^ QI or PDSA as a 'black box' intervention. 

There is an urgent need for greater scientific rigour 
in the application and reporting of these methods to 
advance the understanding of the science of improve- 
ment and efficacy of the PDSA method. The PDSA 
method should be applied v\^ith greater consistency 
and v\^ith greater accordance to guidelines provided by 
founders and commentators^^ 3 o 44 45 
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